Integrated Personal Information Management System

ABSTRACT

An integrated personal information management system is described. In one embodiment, for example, an integrated personal information management system is described that comprises: a receptacle holding at least one paper page that is capable of receiving pen strokes from a digital pen; a digital pen for recording pen strokes when a user writes on a particular paper page; a page identifier for identifying the particular paper page on which the user is writing; and a handheld personal information device in communication with the digital pen and page identifier for processing the recorded pen strokes associated with the particular paper page and displaying the processing results on a display screen.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is related to and claims the benefit of priorityof the following commonly-owned, presently-pending provisionalapplication(s): application Ser. No. 60/481,106 (Docket No. SL/0001.00),filed Jul. 18, 2003, entitled “Integrated Personal InformationManagement System”, of which the present application is anon-provisional application thereof. The disclosure of the foregoingapplication is hereby incorporated by reference in its entirety,including any appendices or attachments thereof, for all purposes.

COPYRIGHT STATEMENT

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

APPENDIX DATA

Computer Program Listing Appendix under Sec. 1.52(e):

This application includes a transmittal under 37 C.F.R.

Sec. 1.52(e) of a Computer Program Listing Appendix. The Appendix, whichcomprises text file(s) that are IBM-PC machine and Microsoft WindowsOperating System compatible, includes the below-listed file(s). All ofthe material disclosed in the Computer Program Listing Appendix can befound at the U.S. Patent and Trademark Office archives and is herebyincorporated by reference into the present application.

Object Description: SourceCode.txt, created: Dec. 11, 2003, 9:23 am,size: 47.1 KB; Object ID: File No. 1; Object Contents: Source Code.

BACKGROUND OF INVENTION

1. Field of the Invention

This invention relates generally to personal information managementsystems and more particularly to an integrated personal informationmanagement system including a paper planner, a personal digitalassistant (PDA), and a digital pen.

2. Description of the Background Art

The term “personal information manager” or “PIM” is often used to referto software or other tools or systems that are used to manage anindividual's calendar, contact information (address/phone book), tasklists, and notes. In this document, the term “personal informationmanager” or “PIM” is used in a broader sense to indicate any tool orsystem that an individual uses to help with the management of his or hercalendar, contact information, notes, task lists, and other personaland/or business information.

There are three broad categories of PIMs on the market today. The firstcategory is paper-based planners (sometimes referred to herein simply as“planners”). Planners are traditional paper-based calendar and contactmanagement systems, as exemplified by the Franklin Planner system, theDay-Timer System, and the Day Runner planner system.

The second category is personal digital assistants, or PDAs, which aresmall handheld computers, typically having a touch-sensitive screen anda stylus which is used for input. The term “PDA” (or “personal digitalassistant”) generally applies to a stand-alone handheld computing deviceused primarily for managing calendar, contacts, tasks, notes, e-mail,and the like. Examples of PDAs are the Palm computer from Palm, Inc. andthe iPAQ Pocket PC from Hewlett Packard. PDAs can be used to store,display, and/or manipulate various personal information including, butnot limited to contact information, calendar information, and the like.Such information can be downloaded from other computer systems, or canbe input by way of a stylus on a pressure sensitive screen of the PDA orby way of other types of input devices such as mechanical keyboards orvoice recognition modules. More recently, a new generation of cellphones with certain PIM capability has been introduced into the market.As these new cell phones (known as “smartphones”) include many of thefeatures of PDAs, for purposes of this discussion they are also includedin the PDA category of personal information managers. In the followingdiscussion, the term PDA is used in a broad sense to include traditionalpersonal digital assistants as well as similar computing devices(including cell phones with PDA features) and associated accessories andstructures.

The third category of PIM is a personal computer (PC)-based solution. Inthis case a desktop or laptop computer together with associatedapplication software such as Microsoft Outlook is utilized to providefunctionality similar to that described above as provided by a PDA. Thisfunctionality includes the ability to store, manipulate, and displaycontact information, calendar information, and the like on the PC. ThePC or laptop supporting the PIM application is typically significantlylarger than a PDA in terms of physical size and also employs greater CPUand memory resources. However, PC-based PIM systems also have many ofthe same limitations of PDA-based systems as described below. They alsohave other disadvantages such as increased size, price, and complexity.

Each of these categories of personal information management systems hasits own distinct advantages and disadvantages, and when selecting asystem to use for personal information management, consumers find thereis no perfect solution. The advantages and disadvantages of each typewill now be briefly summarized.

Traditional paper planners have several advantages, including comfort,simplicity, and familiarity. Using a pen to record information on apiece of paper is an input process that everyone knows. Becausepractically everyone has written with pen and paper since childhood,there is nothing new to learn in order to use a traditional planner. Thetraditional planner provides a simple, familiar, comfortable environmentthat imposes a minimal impediment to the flow of ideas. Traditionalplanners also offer flexible, personalized use. Planners tend to becomevery personalized documents as each user develops his or her ownparticular style. A user may include notes in the margins, diagramsand/or annotations in the notes area, or special symbols next to phonenumbers to differentiate friends from clients. He or she may also writeat angles, circle or star important items, or include receipts,newspaper clippings, or letters tucked into the pockets or between thepages of the planner. While the printed forms provide structure andguidelines for usage, users are free to customize and improvise. Theplanner enables a user to be creative and to personalize the tool,rather than requiring the user to adapt to the information managementtool.

A traditional planner is also easy to navigate. Users are accustomed tothe process of flipping through pages and intuitively understand that alarger number of pages corresponds to a larger number of days. Thefamiliarity, flexibility, and ease of use of planners are some of thetraditional advantages of these tools compared to more recent PDA andPC-based technologies.

These advantages are, however, offset by several disadvantages. First,the traditional planner typically provides no safety net to the user.Losing one's planner is nothing short of traumatic for serious users. Auser rarely, if ever, keeps a “backup” of his or her planner.Accordingly, if the planner is lost or stolen, all of the phone numbers,addresses, dates, lists, plans, ideas and other information retained inthe planner are irretrievably lost in most cases. This wrenchingexperience leaves the user with no choice but to start from scratch,reconstructing data from memory, and probably never recovering all ofthe information. For a user storing personal information in the planner,this can be extremely upsetting and annoying. For one storing businessand career information in the planner, the loss can also havesignificant financial repercussions as valuable and irreplaceablecontacts and plans are often lost with the planner.

A paper planner also provides only limited search capability. Itprovides no easy way to find a particular phrase or item of informationif the user does not recall the date or some visual cue about the pagethe data is on. The paper planner also provides little or no automation.Recurring events must always be entered by hand, leaving open thepossibility of losing or forgetting birthdays, anniversaries, or otherimportant dates, appointments, or meetings. In addition, one can onlycomfortably fit a limited number of calendar pages into a planner, soaccessing information from the past or from future dates (e.g., dates inthe following year) is difficult if not impossible. The planner alsoprovides no automated task management—tasks not completed on a given daymust be manually carried forward to the next day. It also does notfacilitate sharing of information. For example, there is no easy way toview a spouse's calendar, and no way for an administrative assistant toview or modify a user's calendar. Additionally, the paper plannerprovides no way to cut and paste information between the calendar ande-mail. Given that so many people regularly work with personal computersand other automated devices, having to manually track everything in apaper planner is inefficient, inconvenient, redundant, and timeconsuming.

With the advent of PDA- and PC- based PIMs, a number of these problemshave been addressed. Users can backup data from their PDA or PC to othersystems, thereby guarding against the loss of this data. PDAs and PCsalso provide search functions that enable a user to find any word in anyof their data as well as other automated features includingautomatically distributing recurring events throughout the calendar, andautomatically carrying forward incomplete tasks to the next day. PDA-and PC-based calendars can also be shared, and may include alarms toremind the user of events. However, these new features are providedtogether with a new set of problems.

The process of entering data into the PDA is one of the main complaintsof PDA users. Few users find the tactile experience of using a stylus ona plastic screen to be enjoyable. But this is a minor point compared tothe inefficiency and inconvenience of having to enter text using anartificial writing system like Graffiti, or having to peck out letterson a tiny on-screen keyboard or add-on keyboard. Although variousefforts have been made to improve data entry features of PDAs, userscontinue to find the data entry process to be complex, difficult, andgenerally unpleasant.

PC-based PIM systems provide a partial solution to some of these dataentry problems. For example, the Tablet PC from Microsoft addresses someof these data entry problems by enabling a user to write with a stylusin his/her own handwriting on a larger screen. This improves theexperience of interacting with the computer via a stylus to some extent.However, the user is still left with the unpleasant tactile experienceof writing with plastic on plastic. Also, the user is required to writelarger on the screen than they normally would write on a piece of paperbecause the lines formed on the screen are fatter (i.e., thicker) thanthe line left by a normal pen, and the handwriting recognition softwareworks poorly with smaller writing. Worse still, the user must learn tonavigate a complex, multi-modal “Input panel” so that the Tablet PC candiscern handwriting input from cursor movements, “right clicks”, andgesture commands. For these reasons, writing on a Tablet PC does notcompare favorably to the easy and familiar process of writing on paperwith a pen. Tablet PCs (as well as laptop computers and other types ofPCs) are also usually much bigger and heavier than planners or PDAs. Infact, a general disadvantage of all tablet, laptop, or desktop PC-basedsolutions is that these devices are typically heavier and less portablethan either PDAs or paper planners and are also typically more complexand more expensive.

PDA- and PC-based PIMs offer very little flexibility. One generally mustuse the PDA or PC and the accompanying software in the way in whichthese components were designed. T here is little or no opportunity topersonalize the capabilities of the PDA or PC. For example, a usercannot make notes in the margins, fold over the corners of importantpapers, tuck other papers between pages, or circle important items. PDA-and PC- based PIMs typically provide a rigid set of features to be usedin accordance with specified procedures. Moreover, these features can becumbersome to navigate. Instead of intuitive, tactile pages and tabs,navigation on a PDA or PC must be accomplished by clicking crypticon-screen icons or by using various combinations of buttons.

While the advent of PDA- and PC-based PIMs resolved many of theshortcomings of the paper planner, some advantageous features were lost,and several new problems were introduced. Today the PIM market remainsdivided, and the question of which PIM solution is best has nodefinitive answer. Each of the current approaches is subject tosignificant problems and limitations.

One recent attempt to address some of the shortcomings of current PIMsis the Franklin Covey iScribe, which is intended to work with an opticaldigitizing pen from Logitech called the “lo”. This product combinationallows users to write on special paper using a special ballpoint pen,and transfer a digitized version of their writing back to a PC runningiScribe software. The iScribe software, in turn, provides integrationwith Microsoft Outlook such that users can view within Outlook theinformation which they entered on paper in their planner. Although thisdesign does allow users to write on paper, the iScribe product does nottruly integrate digital and paper planning, but rather creates a limitedone-way link from the paper domain to the digital domain. No handwritingrecognition capability is provided, and even if a handwritingrecognition application was used on the PC, the user has no feedbackwhile writing about whether the handwriting recognition application onthe PC will correctly interpret what they are writing. Thus, foreverything the user has written, the process of reviewing the resultingrecognized text is a distinct extra step which must be performed at thePC. This is inconvenient and time-consuming. Also, the planner providesno access to information that resides in the digital domain; users canmerely transfer what they have written in their planners into thedigital domain (e.g., into Microsoft Outlook on the PC).

The Seiko SmartPad takes yet another approach, allowing a user to inserthis or her PDA into a folio which includes a paper notepad and a specialpen which can digitize what is written on the pad. Information writtenon the pad is transferred into the PDA in graphic form such that it canbe attached to events or contacts, or simply saved as individualdocuments. This approach allows the user to have access to digital datavia the PDA interface, and to enter freehand drawing data into the PDAusing pen and paper. But here again, there is only very limitedintegration of the paper and digital domains. The SmartPad productintegrates a blank pad with a PDA, instead of integrating a fullyfunctional paper planner with a PDA. Thus, instead of a hybrid productwith the best features of PDAs and paper planners, the product offersonly a rudimentary capability to use the paper to enter drawings intothe PDA. With the SmartPad, there is no way to create new events orcontacts working from paper. The user has only a small lined notepad onwhich to enter data, and there is no handwriting recognition capabilityprovided. Also, the SmartPad device has no ability to discern one pagefrom another. In addition, the user must write in an artificially largestyle in order to produce legible results on the PDA. There is also noway to control the PDA from the paper. Therefore, the user is forced toalternate between using a stylus on the PDA screen and using a pen onthe paper pad.

What is needed is a tightly integrated solution that combines the bestfeatures of each of these existing solutions. The solution should allowthe user to enter data in his or her own handwriting, performhandwriting recognition on the data entered, provide quick feedback onthe recognition process, and allow the user to instantly make any neededclarifications or corrections. It should allow the user to enter newdata, such as appointments, contacts, tasks, and notes in a familiar,simple, and intuitive way based on the already-familiar paper plannerusage model, and should provide intuitive navigation through the datathat is entered. The solution should also provide full access to theuser's digital data, such as documents, email, and events entered via aPC or PDA instead of via the paper and digital pen. At the same time, itshould enable data to be shared with others, integrated with otherdevices, and automatically backed up at regular intervals. The solutionshould also automate normally tedious and error prone processes andshould provide for automated search capabilities. Ideally, the solutionshould, to the greatest degree possible, accommodate the working styleand systems of the user and allow for personalization of the PIM system.It would also be advantageous if the solution cost less and were morerugged than a PC or Tablet PC. The present invention provides a solutionfor these and other needs.

SUMMARY OF INVENTION

An integrated personal information management system is described. Inone embodiment, for example, an integrated personal informationmanagement system, the system of the present invention is described thatcomprises: a receptacle holding at least one paper page that is capableof receiving pen strokes from a digital pen; a digital pen for recordingpen strokes when a user writes on a particular paper page; a pageidentifier for identifying the particular paper page on which the useris writing; and a handheld personal information device in communicationwith the digital pen and page identifier for processing the recorded penstrokes associated with the particular paper page and displaying theprocessing results on a display screen.

In another embodiment, for example, a method of the present invention isdescribed for displaying information at a portable device responsive touser input on a paper page, the method comprises steps of: providing atleast one paper page, each paper page capable of receiving pen strokesmade using a digital pen; recording pen strokes made by a user on apaper page using the digital pen and a page identifier identifying thepaper page; transferring the pen strokes and page identifier to theportable device; and generating information for display at the portabledevice based on processing the pen strokes and the page identifier.

In yet another embodiment, for example, a portable personal informationdevice, the device comprising: at least one paper page, each paper pagecontaining printed areas for receiving pen strokes when a user writesusing a digital pen; a page identifier for identifying the paper page onwhich the user is writing; a digital pen for recording pen stroke datawhen a user writes on the paper page and transferring the pen strokedata to a processing module; a processing module for generatinginformation for display based on processing the pen stroke data and thepage identifier; and a display module for displaying informationgenerated by the processing module.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an illustration of the currently preferred embodiment of theintegrated personal information management (PIM) system of the presentinvention.

FIG. 2 is a block diagram of the components of the currently preferredembodiment of the integrated personal information manager system.

FIG. 3 is a general block diagram of the PDA component of the currentlypreferred embodiment.

FIG. 4 is a block diagram of a software system that is provided fordirecting the operation of the PDA component.

FIG. 5A is a block diagram of the components of the handwritingdigitization system of the currently preferred embodiment of the system.

FIG. 5B is an illustration of an example paper calendar pageillustrating the format of a planning page which can be used inconjunction with the present invention.

FIG. 6A is a high-level block diagram of the components used in the pagesensing operations of the present invention.

FIG. 6B is a block diagram of the page sensor interface electronics ofthe currently preferred embodiment.

FIG. 7 is a block diagram illustrating the software components of thecurrently preferred embodiment of the present invention.

FIGS. 8A-B comprise a single flowchart illustrating a sequence ofoperations in adding a new event to the calendar of the system of thepresent invention in response to user input.

DETAILED DESCRIPTION

Glossary

The following definitions are offered for purposes of illustration, notlimitation, in order to assist with understanding the discussion thatfollows.

Bluetooth: “Bluetooth” refers to a short-range radio technology aimed atsimplifying communications among devices and between devices and theInternet. It also aims to simplify data synchronization between devicesand other computers. Products with Bluetooth technology must bequalified and pass interoperability testing by the Bluetooth SpecialInterest Group prior to release. The Bluetooth 1.0 specification, thedisclosure of which is hereby incorporated by reference, consists of twodocuments: (1) the Foundation Core, which provides designspecifications, and (2) the Foundation Profile, which providesinteroperability guidelines. The Bluetooth specifications are availablevia the Internet (e.g., currently at www.bluetooth.com).

PDA: “PDA” or “personal digital assistant” generally refers to astand-alone handheld computing device used primarily for managingcalendar, contacts, tasks, notes, and e-mail, exemplified by the Palmcomputer from Palm, Inc. and the Pocket PC iPAQ from Hewlett Packard.PDAs can be used to store, display, and/or manipulate various personalinformation including, but not limited to contact information, calendarinformation, and the like. Such information can be downloaded from othercomputer systems, can be input by way of a stylus and/or pressuresensitive screen of the PDA, or can be input using other types of inputdevices such as a mechanical keyboard or a voice recognition module.More recently, a new generation of cell phones include many of thefeatures of PDAs. In this document, the term PDA is used in a broadercontext to include traditional personal digital assistants as well assimilar computing devices (including cell phones with PDA features) andassociated accessories and structures.

PIM: The term “PIM”, which stands for “personal information manager”, isoften used to refer to hardware, software, or other tools or systemsthat are used to manage an individual's calendar, contact information(address/phone book), task lists, and notes. In this document, the term“personal information manager” or “PIM” is used more broadly to refer toany tool or system that an individual uses to help with management ofhis or her calendar, contact information, task lists, notes, and otherpersonal and/or business information.

USB: USB stands for Universal Serial Bus, an external bus standard. TheUniversal Serial Bus is a bi-directional, isochronous, low cost,dynamically attachable serial interface. A single USB port can be usedto connect up to 127 physical devices, such as computer mice, modems,digital cameras, scanners, and keyboards.

Introduction

Referring to the figures, exemplary embodiments of the invention willnow be described. The following description will focus on the presentlypreferred embodiment of the present invention, which is implemented inhardware (e.g., PDA, digitizing pen, paper, and page sensor) andsoftware (e.g., driver, application, or the like) running under anoperating system, such as the Microsoft Windows CE operating system. Thepresent invention, however, is not limited to any one particularapplication or any particular environment. Instead, those skilled in theart will find that the system and methods of the present invention maybe advantageously embodied on a variety of different platforms,including PalmOS, Embedded Linux, Symbian, Macintosh, Linux, Solaris,UNIX, FreeBSD, and the like. Therefore, the description of the exemplaryembodiments that follows is for purposes of illustration and notlimitation.

The exemplary embodiments are primarily described with reference toblock diagrams or flowcharts.

Overview of Integrated Personal Information Management System

The present invention provides a personal information management systemthat combines the best aspects of a paper-based planner, a digital pen,and a PDA in a single integrated unit. In its currently preferredembodiment, the integrated personal information management (PIM) systemof the present invention includes a detailed paper planner integratedwith the other system components in a leather and/or plastic case.

FIG. 1 is an illustration of one embodiment of the integrated personalinformation management (PIM) system 100 of the present invention. Asshown, components of the PIM system 100 are included in a case or folder150. On the left side of the folder 150 is an integrated PDA 110 with atouch-sensitive (pressure-sensitive) LCD display panel 115. Also shownat the left side of the folder and connected to the PDA 110 is a digitalpen 170. A single digital pen is shown in two different locations 170 a,170 b at FIG. 1. In an alternative embodiment, a separate PDA or otherhandheld personal information device (e.g., a “smartphone” or otherportable computing device) may be used in conjunction with the othersystem components. In this alternative, the separate PDA or handheldpersonal information device may be connected to the unit by clipping orsnapping the device into the folder, case or other receptacle. In thepresently preferred embodiment, components of the PIM system 100 areincluded in a case or folder; however, the user of a folder or case isnot required and a different receptacle or attachment means may beutilized for attaching some or all of the components of the system, asdesired. It should also be noted that is not required that the PDA orother handheld personal information device be physically attached to thefolder, case, or receptacle. A number of the components of the PIMsystem 100 (e.g., the digital pen 170 and the PDA 110) are portable andmay be detached and/or reattached from the folder, case, or receptaclefrom time to time and various attachment means may be used forconnecting system components.

At the right side of the PIM system 100 is a binder 120 containing paperplanning pages 130. The binder 120 incorporates an (optional) pagesensor (optical sensor) 140 which in the presently preferred embodimentis mounted on a stem on the top of the binding as shown at FIG. 1. Thepage sensor 140 is positioned and aimed such that it can read a barcodeprinted on one of the exposed planning pages 130, and thereby determinethe current page. The page sensor 140 is connected to the PDA 110, andthe current page information is used by the PDA 110 to show informationrelevant to the current page on the LCD display panel 115. For acalendar page, this might include recurring events, tasks due, or aspouse's calendar for that same date. In an alternative embodiment thebinder and/or page sensor may be omitted. Those skilled in the art willappreciate that various alternative receptacles or attachment means maybe used for holding or attaching the paper pages (and other components)in lieu of a binder. In addition, if the page sensor is not included,the current page to be displayed on the display panel may be determinedby other detection means. For example, a user can indicate the currentpage that is open in the planner by touching the page with the digitalpen 170.

As the user writes on one of the planning pages 130 with a digital pen170, everything written is digitized and transferred from the digitalpen 170 to the PDA 110. Because the digital pen 170 communicates withthe PDA 110, everything the user writes is recorded in digital form, butthe familiar pen and paper interface is used for recording theinformation. The PDA 110 provides the capabilities of an independentPDA, including automation and navigation not provided by a paperplanning system.

The system combines the benefits of paper-based planning and PDA-basedplanning in a single unit. The PDA and the paper planner together form acomposite view of the user's data, which may have been written on thepaper using the digital pen, or may have been synchronized to the PDAfrom an external source. The page sensor 140 allows the PDA 110 to trackthe currently open page, enabling the PDA to automatically showinformation relevant to that page on its LCD display panel 115. Thepresently preferred embodiment of the system includes a touch-sensitive(pressure-sensitive) LCD display panel 115 which allows simple,intuitive navigation of data. The system is automated, providingpowerful search capabilities and enabling data to be automaticallybacked up at regular intervals. It supports recurring events andautomatically carrying forward incomplete tasks. For example, the usercan touch text they have previously written, and this action causes asearch to be initiated for related notes and documents. The system canbe integrated with personal computers and other devices, providingsynchronization with PIM information stored on those systems. The systemis flexible and allows users to personalize their PIM and develop theirown style of usage.

System Components

Overview of System Components

FIG. 2 is a block diagram of the components of the currently preferredembodiment of the integrated personal information manager system 100 ofthe present invention. As shown, the integrated PIM system 100 includesa PDA (or similar) computing system 110, a digital pen 170, a pagesensor 140, and paper planning pages 130. The PDA (or PDA-like computingsystem) 110 includes software programs 220, a USB (Universal Serial Bus)port 203, a Bluetooth radio 204, and a page sensor interface 205. Alsoshown at FIG. 2 are optional devices which can be used with the systemof the present invention. These include a personal computer (PC) 280 anda printer 290 connected to the PDA 110 via the USB port 203 and/or theBluetooth radio 204. The components of the PIM system 100 and theiroperations are described in more detail below.

The PDA-like computing system (or simply “PDA”) 110 of the currentlypreferred embodiment of the present invention is based on a PDAdevelopment platform such as that available from Intel Corporation asproduct number DBPXA255A0. Alternatively, Sophia Systems' PXA255Sandgate Board can also be used for implementation of the PDA componentof the present invention. The PDA 110 of the currently preferredembodiment includes several hardware and software components which aredescribed in greater detail below. Those skilled in the art willappreciate that the present invention may also be implemented usingvarious other portable devices, including PDAs, “smartphones”, and otherpersonal information devices.

The digital pen 170 of the currently preferred embodiment is an opticaldigitizing pen providing handwriting digitization capability. The systemof the present invention leverages the recent development of digital pentechnology capable of digitizing what is written in ink on a piece ofpaper. An example of this handwriting digitization technology is pentechnology from Anoto Group AB of Sweden. More particularly, the digitalpen component employed in the currently preferred embodiment is theNokia Digital Pen SU-1B supplied by Nokia of Finland. Furtherdescription of the Nokia Digital Pen SU-1B is available from Nokia andis available via the Internet (e.g., currently atwww.nokia.com/nokia/0,4879,5798,00.html). Further description of theunderlying Anoto digital pen technology is available from Anoto and isavailable via the Internet (e.g., currently at www.anoto.com). It shouldbe noted that although the presently preferred embodiment describes theuse of a Nokia digital pen based on pen technology from Anoto, the useof this particular technology is not required for implementation of theinvention. Instead, the invention may also be implemented using a numberof other forms of systems for digitizing writing with a pen or otherwriting utensil. For example, in an alternative embodiment thehandwriting digitization system may comprise a pen and a separatetracking system that monitors the motion of the pen.

In operation, as a user writes on the paper planning pages 130 using thedigital pen 170, the motions of the digital pen 170 are transmitted tothe PDA 110 via a wireless or wired interface, such as the Bluetoothshort-range radio interface 204 illustrated at FIG. 2. It should benoted that in the event that the digital pen 170 that is employed doesnot support real-time transfer of data to the PDA, the user may need tofirst check a “send” box on the planning page to initiate the transferof pen strokes and associated information to the PDA. As the user writeson a particular planning page, the page sensor 140 also detects thecurrent page and makes this information available to the softwareprograms 220 running on the PDA 110 through the page sensor interface205. The page sensor 140 allows the PDA 110 to track the currently openpage, enabling the PDA's display to be automatically updated to showinformation relevant to the current planning page.

The PDA 110 also provides for connectivity to other devices enablingdata in the PDA to be shared and/or synchronized with other devices. Asshown at FIG. 2, the PDA 110 can be connected to a personal computer(PC) 280 through either a USB interface port 203, or the Bluetooth radio204. Information may be transferred between the PDA 110 and the PC 280through these interfaces. The PC 280 may also have access to a printer290. The printer 290 may be used to print new planning pages 130 whichmay be inserted into the PIM system 100. Alternatively, the PDA 110 maybe directly connected to the printer (e.g., using USB port 203). Themajor components of the system of the present invention will next bedescribed in greater detail.

PDA Component

FIG. 3 is a general block diagram of the PDA 110 component of thecurrently preferred embodiment. As shown, the PDA 110, comprises acentral processing unit(s) (CPU) and field programmable gate arrays(FPGA) 301 coupled to random-access memory (RAM) 302, a flash read-onlymemory (Flash ROM) 303, a universal synchronous/asynchronous receivertransmitter (USART) 306, a USB port 203, an infrared (IR) port 309, aBluetooth radio 204 providing external wireless connectivity through anantenna 313, input switches 314, a liquid crystal display (LCD) 115, atouch panel 317, audio input/output circuitry 315, a page sensorinterface (XILINX) 205, and a Compact Flash (CF) socket 311 which canaccommodate a variety of Compact Flash (CF) cards. As shown, a CompactFlash card 310 is connected to the PDA via the Compact Flash socket 311.Although not shown separately, a real time system clock is included withthe PDA, in a conventional manner.

In the currently preferred embodiment of the present invention, the CPU301 comprises a processor of the Intel Pentium family of ARM (PXA250)microprocessors. However, any other suitable high speed, low powerprocessor may be utilized for implementing the present invention. TheCPU 301 communicates with other components of the system via abi-directional system bus (including the necessary input/output (I/O)FPGA XILINX (XCR3132) controller circuitry and other “glue” logic). Thebus, which includes address lines for addressing system memory, providesdata transfer between and among the various components. Description ofARM microprocessors and their instruction set, bus architecture, andcontrol lines is available from Intel Corporation of Santa Clara, Calif.

Random-access memory (RAM) 302, serves as the working memory for thePDA. Dynamic RAM (DRAM) of sixty-four megabytes or more is employed inthe currently preferred embodiment. More memory may be used withoutdeparting from the scope of the present invention. The flash read-onlymemory (Flash ROM) 303 contains the operating system (OS) as well as thebasic input/output system code (BIOS), which is a set of low-levelroutines in the Flash ROM 303 that application programs and theoperating system can use to interact with the hardware, to perform suchtasks as reading inputs from the touch panel 317, outputting data to theLCD 115, and so forth. The flash memory (ROM) 303 can also containapplication programs as well as data for long-term storage.

The PDA 110 has common input/output (10) ports consisting of a universalsynchronous/asynchronous receiver transmitter (USART) 306, and auniversal serial bus (USB) 203 providing external connectivity to othersystems and devices through a bottom socket 308 on the bottom of the PDAunit 110. The power supply 305, is also connected through the bottomsocket 308. The power supply 305 powers the unit when plugged into apower converter, as well as charges the batteries 304, including bothmain batteries and backup batteries (not separately shown). When thepower supply 305 is not hooked up, the system runs on the batteries 304.When the main battery is too low to operate the unit properly, thesystem will turn off, and use the backup battery to refresh the RAM 302,so that the contents of the memory will not be lost. The switches 314include a power switch for turning the PDA on and off. The switches 314may also include such additional switches as may be desired for specificimplementations.

In addition to the common 10 ports, the PDA 110 has an infrared port(IR) 309 for infrared communications with other devices (e.g., otherpersonal digital assistants). The PDA also has a Compact Flash socket311 which can interface with a variety of Compact Flash cards 310. ThePDA 110 also contains a Bluetooth radio 204 and an antenna 313, used forcommunication with the digital pen. The Bluetooth radio 204 and antenna313 may also be used for communication with other devices (e.g., adesktop or laptop PC).

The PDA 110 uses a liquid crystal display (LCD) 115 to displayinformation to the user. This LCD 115 is usually backlit, with thecapability of turning off the backlight for battery power saving. TheLCD 115 is touch-sensitive and allows simple, intuitive navigation ofdata displayed on the screen. LCD screens are available from a number ofvendors, including Toshiba and Fujitsu of Japan and Samsung of Korea.The currently preferred embodiment uses a 3.8″ LCD Color Display modelnumber LTM04C380K from Toshiba of Japan.

Data can be input into the PDA 110 in several different ways. The usercan input data into the PDA 110 using the digital pen which is connectedto the PDA through the Bluetooth radio (transceiver) 204. In addition,the user may enter data through the use of the touch panel 317. Data canalso be uploaded into the PDA from other devices (e.g., from a personalcomputer via the USB interface 203).

In basic operation, program logic (including that which implementsmethodology of the present invention described below) is preloaded intoFlash ROM 303 memory during the manufacturing process, for execution bythe CPU 301. During operation of the program logic, the PDA accepts userinput from a touch panel 317, as well as the digital pen 170 through theBluetooth radio 204. The system permits selection of applicationprograms, entry of keyboard-based input, and selection and manipulationof individual data objects displayed on the LCD display panel 115.

The PDA displays text and/or graphic images and other data on thedisplay device 115. The video controller in the CPU 301 outputs to theFPGA, which in turn drives the display LCD 115. A hard copy of thedisplayed information, or other information within the PDA, may beobtained by outputting to a printer through the USB 203 or USART 306ports, or through the Bluetooth radio 204.

In one embodiment, the PDA is connected via a page sensor interface 205and a flex cable to a page sensor. The page sensor (not shown at FIG. 3)is capable of reading a bar code or other identifying mark on thecurrent page opened in the planner. Using the page sensor connectedthrough the page sensor interface 205, the PDA can determine which pagethe user has open and display information appropriate to the currentplanning page. It should be noted that although a page sensor isemployed in the currently preferred embodiment, the current page may bedetermined by other means. For example, in an alternative embodiment,the user can indicate the current page that is open in the planner bytouching the page with the digital pen.

Basic System Software

FIG. 4 is a block diagram of a software system 220 that is provided fordirecting the operation of the PDA component 110. Software system(programs) 220, which is stored in system memory (RAM) 302 and on fixedstorage (e.g., flash ROM 303 as shown at FIG. 3), includes a kernel oroperating system (OS) 410. The OS 410 manages low-level aspects ofcomputer operation, including managing execution of processes, memoryallocation, file input and output (I/O), and device I/O. One or moreapplication programs, such as client application software or “programs”401 (e.g., 401 a, 401 b, 401 c, 401 d) may be “loaded” (i.e.,transferred from fixed storage (flash ROM 303) into memory 302) forexecution by the PDA 110. Certain specific applications employed in thesystem of the currently preferred embodiment are described below in thisdocument.

Software system 220 includes a graphical user interface (GUI) 415, forreceiving user commands and data in a graphical (e.g.,“point-and-click”) fashion. These inputs, in turn, may be acted upon inaccordance with instructions from the operating system 410, and/orclient application module(s) 401. The GUI 415 also serves to display theresults of operation from the OS 410 and application(s) 401, whereuponthe user may supply additional inputs or terminate the session.Typically, the OS 410 operates in conjunction with device drivers 420(e.g., “Winsock” driver—Windows' implementation of a TCP/IP stack) andthe ROM-based system BIOS 430, particularly when interfacing withperipheral devices. The OS 410 can be provided by a conventionaloperating system, such as Microsoft Windows CE available from MicrosoftCorporation of Redmond, Wash. Alternatively, the OS 410 can also be analternative operating system, such as Embedded Linux, PalmOS, Symbian,or the like.

Digital Pen and Paper

FIG. 5A is a block diagram of the components of the handwritingdigitization system 500 of the currently preferred embodiment of thesystem. The handwriting digitization system 500 includes as two of itsmajor components the paper 501 and the digital pen 170. The digital pen170 operates like any ordinary pen using the ballpoint cartridge 505, onany paper or writing area. The ballpoint cartridge 505 is replaceable.However, the pen 170 provides handwriting digitization capability whenit is used with paper 501 that is printed with a unique dot pattern 502.

In operation, when the digital pen 170 is used to write on the paper 501(which includes the dot pattern 502), the pressure sensor 506 detectsthat the pen is in use and signals the micro controller 507. The microcontroller 507 activates the camera 504, and searches for the unique dotpattern 502. If the dot pattern is found, the micro controller 507begins recording the pen's writing movements (“strokes” or “penstrokes”) made by the user in the memory 509. The micro controller 507can also transmit the strokes recorded in the memory 509 through theBluetooth radio 508 (e.g., for transmission to the PDA 110). The pen 170is powered through a battery 510, which can be recharged through a powersupply 511, when connected to its charger through the power jack 512.

In the presently preferred embodiment, the paper 501 is digitallyenabled by a printed pattern of gray dots 502, with a varied spacing ofapproximately 0.3 mm (0.01 inch). The pattern of dots allows dynamicinformation coming from the digital camera in the pen to be processedinto signals representing the pen strokes the user makes on the page,together with the page identification and the location on the page. Thedot pattern of the paper used in the currently preferred embodiment islicensed from Anoto Group AB. Further description of the dot pattern isavailable from Anoto Group AB of Finland, the disclosure of which ishereby incorporated by reference. A description of the dot pattern anddigital pen technology is available via the Internet (e.g., currently atwww.anoto.com).

FIG. 5B is an illustration of an example paper calendar page 550illustrating the format of a planning page which can be used inconjunction with the present invention. This example calendar page 550is a standard paper calendar page and does not include the Anoto dotpattern described above. An example of Anoto-pen enabled planning pageswhich can be utilized in conjunction with the present invention are theiScribe digitally enabled planner forms from Franklin Covey.

As shown the paper calendar page 550 includes a bar code 560. Asdescribed above, the bar code 560 can be read by the page sensor toidentify the current page open in the planner. The paper can also beprinted with various forms which may include special icons and regionsthat trigger functions in the PDA when touched with the pen, such thatthe pen and paper can be used not only to enter new data into the PDA,but also to control PDA program functions. These printed areas of thepaper (e.g., text or icons) act, in effect, like buttons or othercommand elements, and activate functionality in the embedded PDA whentouched with the digitizing pen.

Further, the user can write information on the paper, and then latertouch this written information with the pen to activate functions on thePDA specific to and determined by that written information, effectivelycreating dynamic handwritten command buttons or links. For example, ifthe user taps a calendar event where he or she has previously writtenthe word “Cisco”, the embedded PDA automatically performs a search forall documents, events, and contacts that contain the keyword “Cisco”.

Page Sensor

FIGS. 6A-B are block diagrams illustrating the page sensing componentsand operations of the currently preferred embodiment of the presentinvention. FIG. 6A is a high-level block diagram of the components usedin the page sensing operations of the present invention. As shown, thepage sensing operations rely on the use of several different components,including a barcode (or other icon or identifier) 602 printed on aspecific area on a paper page 601, a page sensor (input apparatus) 140,and the PDA 110.

The page sensor input apparatus 140 includes a lens 613, an opticalsensor 614, a digital to analog (DAC) amplifier 618, and a power supply619. In the presently preferred embodiment, the sensor 614 is an opticalone-dimensional CCD light sensor. In the preferred embodiment, a SonyILX551B CCD light sensor is employed which provides the capability of a2048-pixel resolution, but any equivalent CCD sensor could be used.Utilizing a sensor having a 2048-pixel resolution allows for a minimumof a twelve (12) bit bar code to have over 150 pixels per bar todetermine the difference between a bar coding of a one (1) or zero (0).Thus, a twelve (12) bit bar code allows for over a 4000-page capability.Alternatively, a scanning type bar code reader could be employed insteadof the CCD sensor. A separate power supply 619 generates the requiredvoltages (power) needed by the optical sensor 614. In the currentlypreferred embodiment, the power supply uses the PDA power supply 305(illustrated at FIG. 3) as its power source.

In operation, the lens 613 focuses the optical sensor 614 onto an areaof the paper page 601 where the barcode 602 is printed. As previouslyshown at FIG. 1, in the currently preferred embodiment the page sensorinput apparatus 140 is mounted in a fixed location above a specificportion of the planning pages included in the PIM system 100. The lens613 of the input apparatus is mechanically positioned so that theoptical sensor 614 can view the barcode area of the paper page 601.

The input apparatus 140 is electronically connected to the PDA throughthe flex cable 625, and interfaces to the PDA 110 through the pagesensor (XILINX FPGA) interface 205. The page sensor interface (XILINXFPGA) 205, generates timing signals as well as buffers the incoming datafrom the sensor 614, to the CPU 301. The CPU 301 controls the pagesensor 205 through the software 638, by polling the sensor periodicallyto determine if the user has turned to a new page and decoding the barcode written on the page. The page sensor interface components betweenthe input apparatus and the PDA will now be described in greater detail.

Page Sensor Interface

FIG. 6B is a block diagram of the page sensor interface electronics ofthe currently preferred embodiment. As shown at FIG. 6B, the page sensorinterface electronics include three major components, the optical sensor614, the page sensor (XILINX) interface 205, and the CPU 301. Theoptical sensor 614 (e.g., Sony CCD sensor ILX551B) connects to the pagesensor interface 205. Further description of the Sony CCD sensor ILX551Bis available from Sony of Japan. The page sensor interface 205, in turn,interfaces to the CPU 301.

The CPU 301 controls the XILINX chip of the page sensor interface 205,through the address bus 655, control bus 656, and data bus 657. Thecontrol bus 656 consists of chip selects, read, write, and DMA,controls. The XILINX data transceiver 659, as well as the state machine660, and DAC data latch 666, exist in a specific memory block locationset aside for the page sensor, and programmed into the address decodefunction 658. The CPU 301 can access the data from the data input bufferregister 662 through specific addresses within the memory block. Thestate machine 660 can also be programmed by the CPU by outputtingspecific control commands.

Run function: The bit clock 663, is divided down from the master clock654, obtained from the CPU 301. During the read operation “run”, the outgate line 664, is set to high, then the bit clock 663, starts to toggle.The out gate line 664, must have a high setting prior to the rising edgeof the bit clock line, and remain stable until after the last fallingedge of the bit clock line. The clock sequence generally occurs 2105times. During clock pulses 51 through 2099, the 2048 pixel data bits areacquired, by sensing the read data bit line 665, at the rising edge ofthe bit clock line 663. The data input buffer 662, is a 2048 bit shiftregister, capable of holding all 2048 data bits. The state machine 660can be programmed to store any sequential 2048 bits, starting from bit 0in the data stream to bit 57. This allows the machine to capture the“black” pixels as well as the valid pixels, allowing the computer todetermine the noise threshold of the optical sensor 614.

Idle function: During the “idle” mode of operation the state machine 660stops the bit clock 663 to the sensor 614, and enters a state where theCPU 301 can read the data stored in the data input buffer 662. Sixtyfour (64) reads of 32 bits each are required to obtain the full 2048bits of data.

Variable light conditions: Different light conditions can beaccommodated by programming the digital to analog (DAC) amplifier 618,through the parallel data control lines 667, by the DAC data latch 666.A simple multiplying DAC DAC08 is used to amplify the output data streamof the optical sensor 614.

Software Components

In addition to the above-described hardware and software components,several software components or modules are utilized in the system of thepresent invention. FIG. 7 is a block diagram illustrating the softwarecomponents of the currently preferred embodiment of the presentinvention. As shown, the software components include software residingand running on the PDA 110 of the system of the present invention, aswell as software which is designed to reside and run on a computer(e.g., laptop or desktop PC) that can be connected to the PDA 110 aspreviously described.

The software running on the PDA currently consists of six softwarecomponents which together with the above-described components implementcertain core functionality of the system of the present invention. Asshown at FIG. 7, the software components (modules) running on the PDAinclude a core component 701, a user interface module 702, acommunication module 703, a text recognition module 704, a PDA syncinterface 705, and a database 706. These components are in addition tothe operating system (not shown) running on the PDA and previouslydescribed above.

The core component 701 acts as a data flow coordinator, guiding databetween the other modules. The user interface module 702 interacts withthe LCD and touch panel 715 of the PDA 110 to implement the userinterface. The communication module 703 implements the necessarycommunications code to interact with input peripherals such as thedigital pen 170 and the page sensor 140.

The text recognition module 704 implements handwriting recognitionalgorithms which convert digitized handwriting into ASCII text. Ahandwriting recognition module suitable for use in conjunction with thepresent invention is Calligrapher SDK, available from Phatware ofMountain View, Calif. Further description of the Calligrapher SDK isavailable from Phatware and is available via the Internet (e.g.,currently at www.phatware.com). The PDA sync interface 705 contains thecode necessary to provide synchronization services between the embeddedPDA and a desktop PC. In the presently preferred embodiment, the PDAsync interface 705 is provided by a Windows CE implementation ofMicrosoft ActiveSync. The database 706 implements a database forstoring, retrieving, and searching data objects handled by the system ofthe present invention, which typically include calendar events,contacts, tasks, notes, expense reports, and application-specific datafor user applications. In the currently preferred embodiment, thedatabase 706 is provided by Microsoft's Windows CE-based Pocket OutlookObject Model, or “POOM”.

As also shown, additional software components may be installed on alaptop, desktop, or server computer for use in conjunction with thesystem of the present invention. As shown at FIG. 7, a desktop syncinterface 717, and a desktop PIM application 718 may be used inconjunction with the present invention. The desktop sync interface 717implements the PC side of the synchronization services, in a fashioncompatible with the PDA sync interface 705. In the preferred embodiment,the desktop sync interface 717 is provided by the Microsoft Windowsversion of Microsoft ActiveSync. Together, the PDA sync interface 705and the desktop sync interface 717 implement a complete synchronizationprotocol and system capable of detecting changes made to the data stores(or databases) on their respective systems, and synchronizing these datastores. The desktop PIM 718 is a complete desktop PIM software packagewhich includes a user interface and a database. In the preferredembodiment, the desktop PIM 718 is Microsoft Outlook.

System Features

General Features

The system enables a user to enter data entirely with pen on paper, themost comfortable and familiar method for most users. Because the inputmethod is pen on paper, personalization capability is also preserved. Auser can make notes in the margin, draw pictures on the page, or add apersonal coding scheme. The PDA component of the system records anddisplays everything written by the user. A user may navigate simply byflipping through pages in the planner portion of the system. The PDAdetects the current page using the page sensor, and can automaticallyfollow the user's actions and display information appropriate to thecurrent page.

While facilitating easy data entry and enabling a user to personalizethe data that is entered, the system of the present invention alsoprovides the advantages of a PDA. First, the impact of loss of the paperplanning pages is reduced because data can be transferred to otherdevices (e.g., backed up frequently to a personal computer). The systemalso provides the automation advantages of a PDA. A user can perform asearch by flipping through pages, by touching a search term and a searchicon on the paper with the digitizing pen, or by entering searchkeywords in the display panel. The key word search provides access toall of the data in the system. In addition, recurring events do not needto be manually re-entered time and time again. A user can just turn tothe appropriate page, and the display indicates the recurring eventsthat occur on that day. Carrying forward incomplete tasks is alsoautomatic.

The system also integrates with other devices, enabling data to beshared with others and synchronized. For example, a user may share acalendar from his or her planner in powerful new way, with a secondcalendar shown on the display panel for comparison. In addition, datacan be saved to a personal computer and integrated with popular programslike Outlook using synchronization software such as Microsoft'sActiveSync, Palm Inc.'s HotSync, or Pumatech Inc.'s Intellisync.Conversely, new data can be entered on the PC and synchronized back tothe PDA component of the system of the present invention. Any new dataadded this way would appear in the display panel when appropriate. Anexpanded data scope is also provided; the user can view information thatextends beyond the physical pages present in the planner via the displaypanel.

Smart Forms

The system of the present invention also enables data entry on acomposite medium of paper forms and an automatic, intelligent,integrated PIM-based data entry system. Forms are good for handlingrepeated entry processes that usually involve the same data elements orfields. However forms often quickly become complex and errorprone whenthere are: (1) many data elements that are rarely needed; (2) multipledecision points in the form which redirect the user (e.g., “if youentered a 1 here, go to section B; otherwise, continue to section C”);(3) multiple-choice fields with a large number of options to choosefrom; or (4) data dependencies between fields which must be enforced.These types of complexity-inducing entry types and decision points arereferred to herein as “complex form data”.

With a hybrid paper form/electronic entry system, the paper formincludes only the most commonly-used fields, thereby remaining clean,simple, and focused. When there is complex form data to be gathered,these more complex portions can be handled by the PDA, keeping the paperform simple and clean. The PDA of the system of the present inventioncan be used to supplement and enhance the paper form in several ways. Ifthe user enters an “invalid value” in the form, the PDA alerts the userwith an audible error tone and/or error message. The definition of an“invalid value” may be determined dynamically from the values ofprevious entries made on the form. Also, if the user enters a value in a“decision-point” field that impacts the data to be collected next, thePDA can direct the user to the appropriate next form section and verifythat he or she continues in the correct place, producing an error toneand/or message if he or she does not continue in the correct place. ThePDA may also provide other feedback, such as directing the user tocomplete all or part of the rest of the data entry directly on the PDAscreen or to enter the required data in a “soft field” on the form,which is a variable-purpose field labeled with a general identifier suchas “custom 1”, “custom 2”, or the like. In addition, if amultiple-choice field has a large number of possible selections, thesecan be displayed on the PDA screen, saving space on the form. Thus, thehybrid smart form which is enabled by the integrated system of thepresent invention provides dynamic data validation, real-time userguidance, form simplification, and paper savings.

Automatic Association of Notes

The present invention also enables notes written by the user in thepaper planner portion of the system to be automatically associated withthe event corresponding to the current time. For example, a user maybegin to take notes on a blank page at 12:15 pm. The PDA checks thecalendar and finds that this time corresponds to the calendar event“Lunch with Mike”. Subject to optional confirmation, the digitized notesare automatically stored with the event.

Composite Data View

The system of the present invention provides an electronic display whichcomplements and operates in conjunction with a paper calendar (or apaper form of any type) in an integrated fashion. The electronic displaycan display items which have been synchronized from other electronicsources, and the paper and display together form a single, integrated,composite view of the user's data. The display can also supplement thepaper by providing access to data that goes beyond the boundaries of thepaper (e.g., the display can provide access to calendar dates beyond thedate of the last physical page currently carried in the planner).

Detailed Operation

Example of Event Entry Operations

The following sequence of events illustrates the operations of thesystem of the present invention using an example of a user entering anew event in the integrated calendar provided by the system. Thefollowing description presents method steps that may be implementedusing computer-executable instructions, for directing operation of adevice under processor control. The computer-executable instructions maybe stored on a computer-readable medium, such as CD, DVD, flash memory,or the like. The computer-executable instructions may also be stored asa set of downloadable computer-executable instructions, for example, fordownloading and installation from an Internet location (e.g., Webserver).

FIGS. 8A-B comprise a single flowchart 800 illustrating a sequence ofoperations in adding a new event to the calendar of the system of thepresent invention in response to user input. The process begins at step801 with the user opening the planner pages to a particular (calendar)page that includes the desired date for scheduling the event. At step802 the page sensor detects the current page and provides this data tothe core module on the PDA via the page sensor interface and associatedcommunication module.

At step 803 the core module invokes a user interface module to updatethe display (LCD of the PDA) to show data for the current page. Next, atstep 804 the user may make an entry into the current page that is openin the planner. For example, the user may use the digital pen to write“lunch” in the 12:00 pm slot on a particular planner page. As the userwrites the entry, at step 805 the digital pen captures the pen strokesmade by the user and communicates information regarding the pen strokesto the core module on the PDA using the communication module and theBluetooth radio communication channel.

At step 806 the core module on the PDA invokes a user interface moduleto update the display (LCD) to show a digitized form of what the userhas written on the planner page, and sends pen stroke data as a sequenceof vectors to a text recognition module. At step 807 the textrecognition module converts stroke data to text (e.g., in ASCII format)and returns the result back to the core module. At step 808 the coremodule again invokes the user interface module to display the results ofthe handwriting (text) recognition processing on the display (i.e., theLCD of the PDA).

In the event the handwriting recognition results are incorrect, at(optional) step 809, the user can use the digital pen to either re-writetext, or can correct the result by selecting the incorrect words, one ata time, and then selecting from a list of alternate words for each one.When the user indicates that he or she is satisfied with the textgenerated by the system (e.g., by selecting a confirmation box or by theexpiration of a selected time period with no correction activityinitiated), at step 810 the core module sends new event data to thedatabase. This data typically includes date and time of the new event,the associated pen stroke data, and the text results of the handwritingrecognition and correction process.

At step 811 the database creates and stores a new event record. Thedatabase may also (optionally) notify the PDA sync interface that datahas changed. If configured to do so, at step 812 the PDA sync interfacemay automatically synchronize the new data to a desktop PIM (e.g.,Microsoft Outlook running on a connected PC) via the desktop syncinterface described above. Several of these operations will now bedescribed in greater detail.

Receiving Notification from Pen

The following are more detailed examples of several of the operationsthat may be performed in the integrated personal information managementsystem of the present invention. A first example is transferringinformation written by the user with the digital pen to the system's PDAcomponent. The following “OnInitialUpdate( )” method listens for input(e.g., a file) from the system's digital pen:  1: voidCFusionWareEditorView::OnInitialUpdate( )  2: {  3:  4: // ...  5:  6:// ask to be notified of a file change in the PGD targe t directory,which may be a pgd file.  7:  8: // IFF widcomm exists, use it's pathset for incoming files,  9: // else default to temp dir. 10: 11: TCHARbuf[MAX_PATH]; 12: 13: _tcscpy(buf, _T(“\\Temp”)); 14: 15: LPCTSTRlpszWidcommInboxDir = _T(“SOFTWARE\\WIDCOMM\\btConfig\\Services\\0002”);16: CRegKey cKey; 17: LONG IResult = cKey.Open(HKEY_LOCAL_MACHINE,lpszWidcommInboxDir); 18: if (ERROR_SUCCESS = = lResult) 19: { 20: 21:DWORD dwCount = MAX_PATH; 22: cKey.QueryValue(buf, _T(“InboxDirectory”),&dwC ount); 23: cKey.Close( ); 24: } 25: 26: if (_tcslen(buf)) 27: { 28:SHCHANGENOTIFYENTRY cne; 29: cne.dwEventMask = SHCNE_CREATE; 30:cne.pszWatchDir = buf; 31: cne.fRecursive = FALSE; 32: if(!SHChangeNotifyRegister(GetSafeHwnd( ), &cne )) 33: { 34:this−>MessageBox(_T(“Failed to register for pg d change notification!”),g_strAppName); 35: } 36: } 37: // ... 38: }

The PDA component of the system receives information from the digitalpen component. For example, a file may be received from the digital penvia a Bluetooth “push” operation from the pen to the PDA. The aboveroutine provides for a change notification when a new file appears in atarget directory of the PDA.

The system then waits for a change notification indicating that a filehas been received. When a file is received, the following“OnFileChangeInfo” routine is invoked to handle the file:  1: LRESULTCFusionWareEditorView::OnFileChangeInfo(WP ARAM /* wParam */, LPARAMlParam)  2: {  3: FILECHANGENOTIFY *lpfcn = (FILECHANGENOTIFY*)lP aram; 4:  5: FILECHANGEINFO *lpfci;  6: lpfci = &(lpfcn−>fci);  7:ASSERT(lpfci−>wEventId = = SHCNE_CREATE);  8:  9: // this calls code toactually parse the data 10:GetDocument( )−>HandleNewPGD((LPCTSTR)lpfci−> dwItem1); 11: 12:SHChangeNotifyFree(lpfcn); 13: return TRUE; 14: }

Initially, a parser is called to determine if the file that is receivedis a valid file in the appropriate file format (e.g., pgd format). Thefile format, and as a result the parser, are defined by the penmanufacturer, which is currently Anoto. If a valid file is received, thedata can be parsed for use by the system.

Displaying Handwriting on Screen

The data received from the digital pen typically comprises pen strokedata representing the user's handwriting on the paper planner componentof the system. The system provides functionality for displaying thishandwriting on the display screen. For displaying the handwriting, thepen strokes for a given page are first processed as illustrated by thefollowing “SetItemInfo” method:  1: void CEditItemDlg::SetItemInfo(void* pCurrentItem, CPt rArray* pItemsArray, int iCurrentIndex )  2: { 3: for (int i = 0; i < page−>m_drawAreas−>GetSize( ); i+ +)  4: {  5:Pad DrawArea &pdArea = page−>m_drawAreas−>E lementAt(i);  6:  7: intnSize = pdArea.m_userAreas−>GetSize( );  8: for (int j = 0; j < nSize;j++)  9: { 10: PadArea &pArea = pdArea.m_userAreas−>Ele mentAt(j); 11:12: if (pArea.m_strokes) 13: { 14: int numStrokes =pArea.m_strokes−>GetSiz e( ); 15: // add each stroke - probably willchange t his to just pass in the m_strokes list itself instead of 16: //the current one stroke at a time approac h 17: for (int i = 0; i <numStrokes; i++) 18:m_graphicView.AddStroke(pArea.m_strokes−>ElementAt(i) ); 19: } 20: } 21:} 22: 23: void CBitmapDisplay::AddStroke(PenStroke &newStrok e) 24: {25: m_strokes.Add(newStroke); 26: m_bRecreate = TRUE; 27: m_bShowHorz =m_bShowVert = FALSE; 28: }

As shown at line 12, a check is made to see if an area has pen strokesto be processed. If an area has pen strokes to be processed, then an“AddStroke” routine is called to add the pen strokes as provided atlines 17-19. The “AddStroke” routine itself is at lines 23-28.

The following portion of an “OnPaint” method provides for placing thepen stroke data into a memory device context for painting the penstrokes onto the display screen:  1: void CBitmapDisplay::OnPaint( )  2:{  3: CPaintDC dc(this); // device context for painting  4:  5: if(m_bRecreate)  6: CreateMemDC( );  7:  8: if (m_memDC)  9:dc.BitBlt(m_nOffsetx, m_nOffsety, m_sizeClient.cx, m_sizeClient.cy,m_memDC, m_nSourcex, m_nSourcey, SR CCOPY); 10: m_erase = FALSE; 11: }

As shown above, a device context is created for painting and then atline 6 the below “CreateMemDC” method is called. After the memory devicecontext is created (as described below), the above method simply copiesthe bitmap to the current device context for display as provided at line9.

The following “CreateMemDC” method draws the pen strokes to the memorydevice context:

1: BOOL CBitmapDisplay::CreateMemDC( )

2: {

3:

4: if (!m_bRecreate ∥ (!m_bmpNew.m_hObject && !m_p ad Page))

5: return FALSE;

6:

7: if (m_memDC)

8: delete m_memDC;

9:

10: CRect cRect;

151: return !(m_bRecreate=FALSE);

152: }

As provided above, areas (rectangles) of the appropriate size and at theappropriate location for containing the pen strokes are first drawn. Thepen strokes are actually drawn to the memory device context asillustrated at lines 112-145. After all of the pen strokes are drawn,the memory context device is copied for display on the screen asprovided above.

Sending Data to Text Recognition Engine

The system of the present invention includes functionality forconverting a user's handwritten input into text that may be displayed,stored, searched, and otherwise manipulated in the system. The following“ProcessPages” routine receives and processes data from the system'sdigital pen:

1: Ret_t ProcessPages(InstanceID_t id, VoidPtr_t pUserData , PagesPtr_tpContent)

2: {

3: TRACE0(“Enter ProcessPages . . . \r\n”);

4:

5: // create test padfile info

6: CFusionWareEditorDoc *pDoc=(CFusionWareEditorD

281: theMainView→m_progText.SetWindowText(_T(“Wa iting for new pen data. . . ”));

282: return SML_ERR_OK;

283: }

Initially, the pen data is received by the above “ProcessPages” routineas pages from a WBXML parser. Commencing above at line 82, the routineiterates through the pages to process the pen stroke data. The main loopfor converting the pen stroke data commences at line 118. In thepresently preferred embodiment of the system, the pen stroke datacomprises floating point values which are converted to integer valuesfor use by the text recognition module.

After the conversion of the floating point values to integer values, thepen stroke data is associated with particular areas defined by a “PAD”file as provided above at lines 189-219. In particular at line 192, acall is made to the “FindPadArea” method described below to determinethe area in which a given pen stroke was made. After strokes areassociated with particular areas, each area's strokes are sent to a textrecognition module (or engine) as provided at lines 222-270. The textrecognition engine fills in a “RecogData” structure. Each “RecogData”structure contains the strokes, words, and alternate words found by thetext recognition engine and associates each with a page location or“PadArea”. As part of this process the below “DoRecognize” routine iscalled as provided above at line 252.

The below “DoRecognize” routine obtains information from the textrecognition engine about the words that are recognized:  1: BOOLDoRecognize(Calligrapher *m_callig, p_RecogWor dData* data, UINT&num_ans, CFusionWareEditorDoc *pDoc )  2: {  3: TRACE0(“DoRecognizecalled\r\n”);  4:  5: if (!m_callig−>IsLoaded( ))  6: return FALSE;  7:UINT num_alts, num_strokes, weight;  8: TCHAR * ptr;  9: TCHAR str[256];10: #ifdef _DEBUG 11: CString strRecognized = _T(“”); 12: #endif 13: 14:// get the number of words found/recognized 15: num_ans =m_callig−>GetAnswers( CGA_NUM_ANS WERS, 0, 0 ); 16: if (num_ans = = 0)17: return FALSE; 18: 19: (*data) = new RecogWordData[num_ans]; 20: 21:TRACE1(“Answers found: %d\r\n”, num_ans); 22: 23: for (UINT k = 0; k <num_ans; k++ ) 24: { 25: // get number of alternates 26: num_alts =m_callig−>GetAnswers(CGA_NUM_(—) ALTS, k, 0); 27: (*data)[k].numAlts =num_alts; 28: // allocate space to store alternatives 29:(*data)[k].alts = new LPTSTR[num_alts]; 30: // allocate place to storeweights (percent proba bility) for each alternate 31: (*data)[k].weights= new DWORD[num_alts]; 32: 33: // get number of strokes used 34:num_strokes = m_callig−>GetAnswers(CGA_ALT_(—) NSTR, k, 0); 35:(*data)[k].numStrokes = num_strokes; 36: 37: // get the actual strokesused (may or may not m atch those sent in 38: 39: (*data)[k].strokesUsed= (int *)m_callig−>GetAnswers(CGA_ALT_STROKES, k, 0); 40: 41: UINThighWeight = 0; 42: 43: // now ask for each alternative: the first oneis t he one 44: // we default to using 45: for (UINT i = 0, n = 0; i <num_alts; i++ ) 46: { 47: // get the alternate 48: ptr = (TCHAR*)(m_callig−>GetAnswers(CGA_A LT_WORD, k, i)); // Get word alternative49: int len = _tcslen(ptr) * sizeof(TCHAR); 50: (*data)[k].alts[i] = newTCHAR[len]; 51: _tcscpy((*data)[k].alts[i], ptr); 52: 53: // get thisalternates percent probability 54: weight =(int)m_callig−>GetAnswers(CGA_ALT _WEIGHT, k, i); // Get weight of thealternative 55: (*data)[k].weights[i] = weight; 56: if (weight >highWeight) 57: { 58: highWeight = weight; 59: (*data)[k].selIndex = i;60: } 61: 62: 63: // DEBUG ONLY: Place answers in the ans buff er 64:#ifdef _DEBUG 65: strRecognized += ptr; 66: strRecognized += _T(“ : ”);67: wsprintf(str, _T(“%d”), weight); 68: strRecognized += str; 69:strRecognized += “, ”; 70: #endif 71: } 72: #ifdef _DEBUG 73:strRecognized += _T(“\r\n>>>”); 74: wsprintf(str, TEXT(“Nstrokes: %2d”),num_strokes ); 75: strRecognized += _T(“\r\n>>>”); 76: strRecognized +=str; 77: strRecognized += _T(“\r\n>>>”); 78: #endif 79: } 80: 81: #ifdef_DEBUG 82: // TRACE will crash is StrRecognized is too long. 83: CStringtraceOut; 84: if (strRecognized.GetLength( ) > 20) 85: traceOut =strRecognized.Left(20); 86: else 87: traceOut = strRecognized; 88:TRACE1(“%s\r\n”, traceOut); 89: pDoc−>m_szTestData += strRecognized; 90:#endif 91: 92: return TRUE; 93: }

The above “DoRecognize” routine is called after all strokes for a givensession have been sent to the system's text recognition engine. Theabove routine asks the recognition engine to provide the words,alternative words, weights (i.e., probabilities), and actual strokesused for each word recognized by the recognition engine.

The following are the routines that determine the location of a givenpen stroke. These routines determine which page or PAD area a given penstroke is to be associated with by identifying the location (i.e.,bounds) of the stroke and which page or PAD area rectangle contains themajority of the stroke bounds. There are various ways to optimize theseoperations, but the current implementation is straightforward. Anotherresult of the following routines are that the type of item (or object)that the pen strokes create (e.g., an event, task, note, e-mail,contact, etc.) can also be determined by consulting page definitionfiles.

The following “NumRectPointsInRect” function first determines if an areahas any of the bounding vertices:  1: int NumRectPointsInRect(CRect&checkRect, CRect &test Rect)  2: {  3: int count = 0;  4: if(checkRect.PtInRect(testRect.TopLeft( )))  5: count++;  6: if(checkRect.PtInRect(testRect.BottomRight( )))  7: count++;  8: CPointptTopRight(testRect.right, testRect.top);  9: if(checkRect.PtInRect(ptTopRight)) 10: count++; 11: CPointptBottomLeft(testRect.left, testRect.bottom); 12: if(checkRect.PtInRect(ptBottomLeft)) 13: count++; 14: 15: return count;16: }

The above function determines if an area has any of the boundingvertices and returns the number of vertices in the area, if any.

The following “FindPadArea” function obtains the bounds of the strokeand iterates over the PAD areas to see which area, if any, contains mostof the stroke bounds:  1: PadArea *CPadInfo::FindPadArea(CPoint *stroke,UINT n umPoints)  2: {  3: ASSERT(numPoints);  4:  5: // get the boundsof the stroke  6: CRect bounds(stroke[0].x, stroke[0].y, stroke[0].x,stroke[0].y);  7:  8: long &xMin = bounds.left;  9: long &yMin =bounds.top; 10: long &xMax = bounds.right; 11: long &yMax =bounds.bottom; 12: 13: for (UINT i = 1; i < numPoints; i++) 14: { 15: if(xMin > stroke[i].x) 16: xMin = stroke[i].x; 17: else if (xMax <stroke[i].x) 18: xMax = stroke[i].x; 19: 20: if (yMin > stroke[i].y) 21:yMin = stroke[i].y; 22: else if (yMax < stroke[i].y) 23: yMax =stroke[i].y; 24: } 25: 26: double pelsPerUnitx = (.333 *theDoc−>m_pelsPer MM_x); 27: 28: // now adjust to PAD points 29: xMin =PAD_ADJUST(xMin)*pelsPerUnitx; 30: yMin = PAD_ADJUST(yMin)*pelsPerUnitx;31: xMax = PAD_ADJUST(xMax)*pelsPerUnitx; 32: yMax =PAD_ADJUST(yMax)*pelsPerUnitx; 33: // got bounding rect, see what areait is in. 34: PadPage &pPage = this−>m_pages[m_pageIndex]; 35: 36:ASSERT(pPage.m_drawAreas); 37: i = 0; 38: BOOL done = FALSE; 39:PadDrawArea *pdrawArea, *foundDrawArea = NULL; 40: CArray<PadDrawArea,PadDrawArea&> *drawAreas = pPage.m_drawAreas; 41: 42: for (i = 0; i <(UINT)drawAreas−>GetSize( ); i++) 43: { 44: pdrawArea =&drawAreas−>ElementAt(i); 45: if (pdrawArea−>ContainsStroke(bounds)) 46:{ 47: foundDrawArea = pdrawArea; 48: break; 49: } 50: } 51: 52: if(found DrawArea && foundDrawArea−>m_userAre as) 53: { 54: PadArea*pArea, *foundArea = NULL; 55: 56: UINT size =foundDrawArea−>m_userAreas−>Ge tSize( ); 57: for(i = 0; i < (UINT)size;i++) 58: { 59: pArea = &foundDrawArea−>m_userAreas−>El ementAt(i); 60:if (pArea−>ContainsStroke(bounds)) 61: { 62: foundArea = pArea; 63:break; 64: } 65: } 66: 67: if (!foundArea) 68: { 69: TRACE0(“*** Drawarea found, but no user are a found!!!”); 70: return(PadArea*)foundDrawArea; 71: } 72: 73: TRACE1(“Area found name: %s”,foundArea−>m_(—) name); 74: return foundArea; 75: } 76: else 77: { 78:TRACE0(“*** Draw area found, but no user area f ound!!!”); 79: return(PadArea *)foundDrawArea; 80: } 81: }

In particular, at line 60 above a “ContainsStroke” routine is called todetermine if a particular area contains a majority of a given penstroke.

The “ContainsStroke” routine is as follows:  1: BOOLPadArea::ContainsStroke(CRect &bounds)  2: {  3: int count =NumRectPointsInRect(this−>GetRect( ), bo unds);  4:  5: // if it hasmore than two vertices, it must contain al l of the stroke rectangle  6:if (count > 2)  7: return TRUE;  8:  9: // check to see if this areacontain MOST of the boun ds rect. 10: 11: CRect intRect; 12:intRect.IntersectRect(this−>GetRect( ), bounds); 13: DWORD barea =bounds.Height( ) * bounds.Width( ); 14: 15: DWORD iarea =intRect.Width( ) * intRect.Height( ); 16: // TRACE2(“StrokeArea = %ld,IntersectArea = %ld \r\n”, barea, iarea); 17: if ((barea/2) < iarea) 18:return TRUE; 19: return FALSE; 20: }

The above “ContainsStroke” routine determines if a given area contains amajority of the stroke rectangle. First a check is made as shown atlines 3-7 to determine if more than two points of the stroke rectangleare within the area. If the area contains more than two vertices, thisindicates that the area contains all of the stroke rectangle. Theremaining portion of the function commencing at line 11 determines if anarea contains a majority of the stroke rectangle. The function returns“TRUE” if an area contains most of the stroke rectangle and “FALSE”otherwise.

Allowing the User to Select Alternative Words

The results of the text recognition processing described above may thenbe displayed to the user in the user interface (display) of the system.A user may then select particular words and edit them (e.g., in theevent that the words generated as a result of text recognition areincorrect). The system of the present invention also provides fordisplay of available alternative words suggested by the text recognitionengine. In the presently preferred embodiment, when a user selects(e.g., clicks and holds on) an edit control, a context menu is createdwhich is populated with alternative words (if they exist). The following“ContextMenu” routine illustrates the process for creation of thecontext menu:  1: void CEditCtrl::ContextMenu(CPoint point)  2: {  3:CMenu mnuCtxt;  4:  5:  6: if (!m_nMenuID)  7: return;  8:  9:mnuCtxt.CreatePopupMenu( ); 10: 11: // ... 12: 13: // now see if thereare alternate words for word at t his location 14: int x = point.x -GetMarginWidth( ); 15: int y = point.y; 16: CPoint curLoc; 17:MouseToCaret(x, y, curLoc); 18: m_nContextMenuIndex = −1; 19: if(curLoc.y < m_vLines.GetSize( )) 20: { 21: Line &lLine =m_vLines[curLoc.y]; 22: 23: // The wordAlternates structure contains theinfo about alternate words for a given word starting at a given lineindex 24: 25: if (lLine.wordAlternates) 26: { 27: m_ptCaretPos = curLoc;28: 29: CPoint start, end; 30: // from the given word point location, weget t he word bounds 31: GetWordPoints(start, end); 32: 33: WordAltInfo*p_wordInfo = NULL; 34: BOOL bFound = FALSE; 35: // then see if analternate exists for the given word index 36: 37: for (int i = 0; i <lLine.wordAlternates−>GetSiz e( ); i++) 38: { 39: p_wordInfo =&(lLine.wordAlternates−>Elem entAt(i)); 40: int index =p_wordInfo−>index; 41: if (index >= start.x && index <= end.x) 42: { 43:m_nContextMenuIndex = i; 44: break; 45: } 46: } 47: if(m_nContextMenuIndex != −1 && p_wordInf o && p_wordInfo−>word Data &&(p_wordInfo−>wordData−>nu mAlts > 1)) 48: { 49: if (menuAdded) 50:mnuCtxt.AppendMenu(MF_SEPARATOR, 0 , _T(“ ”); 51: 52: RecogWordData*wordData = p_wordInfo−> wordData; 53: 54: for (i = 0; i <wordData−>numAlts && i <= EM_INSERTALTLAST; i++) 55: { 56: if (i !=wordData−>selIndex) 57: { 58: mnuCtxt.AppendMenu(MF_ENABLED,EM_INSERTALT+i, wordData−>alts[i]); 59: menuAdded++; 60: } 61: } 62: }63: MakeCaretVisible( ); 64: } 65: } 66: if (menuAdded) 67:mnuCtxt.TrackPopupMenu(TPM_LEFTALIGN|TPM_(—) TOPALIGN, point.x, point.y,this); 68: }

A call is made to “CreatePopupMenu” to create a pop-up menu as providedat line 9 above. Next, a determination is made as to whether alternativewords are available at the given location by consulting a“wordAlternates” data structure. The “wordAlternates” structure containsinformation about alternative words for a given word starting at a givenline index. The alternative words that are identified c an be displayedto the user.

Additional routines are required in order to keep the alternative wordinformation up to date in the event the user edits any of the items. Thefollowing routines handle most of these operations for trackingalternative words for a given word:

1: BOOL CEditCtrl::InsertLine(LPCTSTR text, int nLength, in t nPosition,WordAltInfoArrayPtr p_wordInfo, int waiIndex)

192: |Line.wordAlternates→Add (p_wordInfo→ElementAt(i));

193: }

194: }

195: else //these seperated for speed's sake, ev en though possible tocombine lines above with this.

196: for (int i=0; i<size; i++)

197: |Line.wordAlternates→Add (p_wordinfo→ElementAt(i));

198: }

199: }

200: // . . .

201: }

If the user cuts, pastes, deletes, or performs other similar actions,these changes must be tracked so that the indexes stay associated withthe correct words, even if such words have been modified by the user. Inthe presently preferred embodiment, an item is only dropped if the firstletter of the word is deleted. However, if the user issues a command to“undo” the deletion of the first letter, the item is re-added. Usually,alternative word information is created when a new line is inserted, asthe code doing the insertion generally passes in an array containingalternative word information.

Constructing an Event

Areas of a paper planning page may be associated with particular events(or objects) such as appointments, tasks, and the like. The followingportion of an “OnInitDialog” routine determines whether an event shouldbe created based on entries made by the user in particular areas of apaper planning page:

1: BOOL CSelectItemsDlg::OnInitDialog( )

2: {

3: // . . .

4: SYSTEMTIME sysTime;

5: CString strText;

6: COleDateTime dt=COleDateTime::GetCurrentTime( ) ;

7: dt.GetAsSystemTime( sysTime);

8:

9: //Is there any “real” data?

10: if (g_padinfo && g_padinfo→m_pages.GetSize( )>0)

11: {

12: UINT nSize=g_padInfo→m-pages.GetSize( );

148: FileTimeToSystemTime(&ft, &sysTime);

149:

150: pNote→SetRecordDT(sysTime);

151: m_ItemsList.AddItem(pNote);

152: }

153: }

154: }

155: }

156: }

157: }

158: }

159: }

The strokes and word data have already been sorted into page and/or PADareas as previously described. These areas themselves may be associatedwith a particular type of event or action (e.g., an appointment or atask object). By entering information in these particular areas of aplanning page, the user may create an event such as an appointment or atask. The above portion of an “OnInitDialog” routine goes through andfinds each area by type. If the user has made entries in the area, theappropriate type of event is created.

Sending Resultant Event to Database

After an event is created, it may be sent to the system's database. Thesystem currently includes a Pocket Outlook Object Model (POOM) database.Each type class, such as “CFusionEvent” for example, has a“WriteToOutlook” function similar to the following “WriteToOutlook”method. This method is called when the user clicks on a “Send toOutlook” button or similar menu item:  1: boolCFusionEvent::WriteToOutlook( void* pOutlookMan ager )  2: {  3: boolbResult = true;  4: CFWPOOMManager* pManager = (CFWPOOMManager*)pOutlookManager;  5: IAppointment *pAppointment = NULL;  6: HRESULThr = S_OK;  7:  8: if( !pManager−>Initialized( ) )  9: return false; 10:11: // Create an event to receive the new data 12: hr =pManager−>GetApplication( )−>CreateItem ( olAppointmentItem, (IDispatch**)&pAppointment; ); 13: 14: // subject 15:pAppointment−>put_Subject((LPTSTR)(LPCTSTR)m_s trSubject); 16: 17: //body/description/notes 18: pAppointment−>put_Body((LPTSTR)(LPCTSTR)m_strBody); 19: 20: // location 21:pAppointment−>put_Location((LPTSTR)(LPCTSTR)m_(—) strLocation); 22: 23:// set all day 24: // LOOK: inputs are assumed in local, if not changethis 25: bool bLocalTime = false; 26: if (m_bAllday) 27: { 28:pAppointment−>put_AllDayEvent(VARIANT_TRUE ); 29: // bLocalTime = false;30: } 31: else 32: { 33: pAppointment−>put_AllDayEvent(VARIANT_FALS E);34: } 35: 36: // start date 37: DATE date = SystemTimeToDATE(m_startDateTime , bLocalTime, pManager ); 38:pAppointment−>put_Start(date); 39: 40: // end date 41: date =SystemTimeToDATE( m_endDateTime, bLoca lTime, pManager ); 42:pAppointment−>put_End(date); 43: 44: // write to store 45: hr =pAppointment−>Save( ); 46: if( FAILED( hr ) ) 47: { 48: goto cleanup;49: } 50: 51: cleanup: 52: 53: if( pAppointment ) 54:pAppointment−>Release( ); 55: 56: if( FAILED( hr ) ) 57: bResult =false; 58: 59: return bResult; 60: }

The above “WriteToOutlook” routine conforms to the Pocket Outlook ObjectModel interface and provides for writing events to the database. Thesystem of the present invention includes a database for storing,retrieving, and searching data objects handled by the system. These dataobjects typically include calendar events, contacts, tasks, notes, andthe like.

While the invention is described in some detail with specific referenceto a single-preferred embodiment and certain alternatives, there is nointent to limit the invention to that particular embodiment or thosespecific alternatives. For instance, those skilled in the art willappreciate that modifications may be made to the preferred embodimentwithout departing from the teachings of the present invention.

1. An integrated personal information management system, the systemcomprising: a receptacle holding at least one paper page that is capableof receiving pen strokes from a digital pen; a digital pen for recordingpen strokes when a user writes on a particular paper page; a pageidentifier for identifying the particular paper page on which the useris writing; and a handheld personal information device in communicationwith the digital pen and page identifier for processing the recorded penstrokes associated with the particular paper page and displaying theprocessing results on a display screen.
 2. The system of claim 1,wherein the paper page includes a particular dot pattern for use inconjunction with the digital pen for recording the user's pen strokes.3. The system of claim 1, wherein said handheld personal informationdevice displays information responsive to input written on theparticular paper page with the digital pen.
 4. The system of claim 1,wherein said handheld personal information device displays informationrelated to a calendar date in response to a user writing on a particularpaper page corresponding to the calendar date.
 5. The system of claim 1,wherein said handheld personal information device includes at least oneinterface for wireless communication with other devices.
 6. The systemof claim 5, wherein the digital pen includes an interface for wirelesscommunication with said handheld personal information device.
 7. Thesystem of claim 1, wherein said handheld personal information deviceconverts pen strokes received from the digital pen to text.
 8. Thesystem of claim 7, wherein the text is displayed on the display screen.9. The system of claim 8, wherein the text is displayed within a fewseconds after the user has written on the particular paper page enablingthe user to identify any necessary corrections to the text.
 10. Thesystem of claim 8, wherein the text is automatically displayed on thedisplay screen of the handheld personal information device withoutrequiring any additional intervening steps.
 11. The system of claim 8,wherein the user may edit the text displayed on the display screen. 12.The system of claim 8, wherein in response to the user selecting textdisplayed on the display screen, alternative words are displayed. 13.The system of claim 8, wherein the text is displayed together with animage representing at least a portion of the particular paper page. 14.The system of claim 1, wherein the particular paper page includes areasfor entering a selected one of appointments, tasks, calendar entries,contacts, and notes.
 15. The system of claim 14, wherein particularinformation is displayed on the display screen in response to a userselecting an area of the particular paper page with the digital pen. 16.The system of claim 14, wherein said handheld personal informationdevice creates a particular type of record in response to a user writingin a particular area of the paper.
 17. The system of claim 16, whereinthe particular type of record comprises a selected one of an appointmentrecord, a task record, an email record, a contact record, and a noterecord.
 18. The system of claim 16 wherein a particular attribute of therecord is defined in response to a user marking a designated area of apaper page.
 19. The system of claim 1, wherein a user may search forinformation stored on said handheld personal information device byselecting an area of the particular paper page using the digital pen.20. The system of claim 19, wherein the user may search for informationby touching a handwritten entry on the particular paper page using thedigital pen.
 21. The system of claim 1, wherein information previouslywritten by the user on the particular paper page on which the user iscurrently writing is displayed on the display screen.
 22. The system ofclaim 1, wherein said page identifier comprises a sensing deviceconnected to the handheld personal information device.
 23. The system ofclaim 1, wherein said page identifier is a component of the digital pencapable of identifying the paper page on which the user is writing. 24.The system of claim 1, wherein the receptacle comprises a folder formounting the paper pages and the handheld personal information device.25. A method for displaying information at a portable device responsiveto user input on a paper page, the method comprising: providing at leastone paper page, each paper page capable of receiving pen strokes madeusing a digital pen; recording pen strokes made by a user on a paperpage using the digital pen and a page identifier identifying the paperpage; transferring the pen strokes and page identifier to the portabledevice; and generating information for display at the portable devicebased on processing the pen strokes and the page identifier.
 26. Themethod of claim 25, wherein the page identifier establishes context forprocessing the pen strokes.
 27. The method of claim 25, wherein saidportable device displays input written by the user on the paper pagewith the digital pen.
 28. The method of claim 25, further comprising:determining a particular area of the paper page containing the penstrokes made by the user.
 29. The method of claim 28, wherein aparticular type of record is created in response to a user writing in aparticular area of the paper page.
 30. The method of claim 29, whereinthe record comprises a selected one of an appointment record, a taskrecord, an email record, a contact record, and a note record.
 31. Themethod of claim 29, wherein a particular attribute of the record isdefined in response to a user marking a designated area of the paperpage.
 32. The method of claim 25, wherein said transferring stepincludes using wireless communication.
 33. The method of claim 25,wherein the portable device comprises a personal digital assistant(PDA).
 34. The method of claim 25, wherein said displaying step includesdisplaying information related to a particular calendar date in responseto a user selecting a paper page corresponding to the particularcalendar date.
 35. The method of claim 25, wherein said displaying stepincludes displaying an image representing at least a portion of a paperpage containing user input entered by the user on the paper page. 36.The method of claim 25, wherein said displaying step includes displayingparticular information based on a particular area of the paper pageselected by the user.
 37. The method of claim 36, wherein saiddisplaying step includes displaying information previously input by theuser in a particular area of the paper page in response to the userselecting the particular area using the digital pen.
 38. The method ofclaim 37, wherein said displaying step further comprises displayinginformation related to the information previously input by the user. 39.The method of claim 25, wherein said displaying step includes displayingfeedback regarding the pen strokes made by the user.
 40. The method ofclaim 25, wherein said generating step includes converting pen strokesreceived from the digital pen to text.
 41. The method of claim 40,wherein the text is displayed to the user.
 42. The method of claim 40,wherein the text is displayed within a few seconds after the user haswritten on the paper page enabling the user to identify any necessarycorrections to the text.
 43. The method of claim 40, wherein the text isautomatically displayed without requiring any additional interveningsteps.
 44. The method of claim 40, wherein the user may edit the textdisplayed at the portable device.
 45. The method of claim 40, wherein inresponse to the user selecting text displayed at the portable device,alternative words are displayed.
 46. A computer-readable medium havingprocessor-executable instructions for performing the method of claim 25.47. A downloadable set of processor-executable instructions forperforming the method of claim
 25. 48. A portable personal informationdevice, the device comprising: at least one paper page, each paper pagecontaining printed areas for receiving pen strokes when a user writesusing a digital pen; a page identifier for identifying the paper page onwhich the user is writing; a digital pen for recording pen stroke datawhen a user writes on the paper page and transferring the pen strokedata to a processing module; a processing module for generatinginformation for display based on processing the pen stroke data and thepage identifier; and a display module for displaying informationgenerated by the processing module.
 49. The device of claim 48, whereinthe paper page includes printed calendar information.
 50. The device ofclaim 49, wherein the paper page includes a selected one ofappointments, tasks, emails, contacts, and notes.
 51. The device ofclaim 48, wherein the paper page includes a particular dot pattern forrecording pen stroke data when a user writes on the paper page using thedigital pen.
 52. The device of claim 48, wherein said digital pentransfers pen stroke data to the processing module using wirelesscommunication.
 53. The device of claim 48, wherein said processingmodule generates information related to a particular calendar date inresponse to a user selecting a paper page corresponding to theparticular calendar date.
 54. The device of claim 48, wherein saidprocessing module retrieves information previously written by the useron the paper page when the user selects the paper page.
 55. The deviceof claim 54, wherein said display module displays the informationpreviously input by the user.
 56. The device of claim 48, wherein saidprocessing module converts pen strokes received from the digital pen totext.
 57. The device of claim 56, wherein the text is displayed by thedisplay module.
 58. The device of claim 56, wherein the text isdisplayed within a few seconds after the user has written on the paperpage enabling the user to identify any necessary corrections to thetext.
 59. The device of claim 56, wherein the text is automaticallydisplayed by the display module without requiring any additionalintervening steps.
 60. The device of claim 56, wherein the user may editthe text displayed by the display module.
 61. The device of claim 56,wherein in response to the user selecting text displayed by the displaymodule, alternative words are displayed.
 62. The device of claim 48,wherein the user may search for information stored on the portablepersonal information device by selecting an area of the paper page usingthe digital pen.
 63. The device of claim 48, wherein the user may searchfor information by touching a handwritten entry on the paper page usingthe digital pen.